Executive Summary

  • Charged with informing the process to develop a debris on debris collision warning threshold to mitigate future collision risk and limit false alarms.
  • The study has focused on the following parameters:
    • Fragment size of at least 1.
    • Five days to time of closest approach (TCA).
    • Concern Events terminology defined as \(\text{Collision Probability} >= 1\text{e-}5\) within a day to TCA.
  • Study progress:
    • We can visualize the missed concern event and false alarm risk space.
    • We have developed an application to explore various warning thresholds.

Problem Formulation

  • Problem Statement: Under what circumstances should debris-on-debris collision warnings be used?
    • What probability of collision should be used as a surrogate for an actual event of concern?
    • What probability of collision should trigger a warning at five days to TCA?
    • How well can we do in warning for events occurring in the future?
  • Mission Statement: Recommend debris collision notification thresholds and develop a mechanism to explore the notification decision space to assess risk of debris-on-debris collisions in space.

Exploratory Data Analysis

  • Questions:
    • How is the Collision Probability distributed?
    • How does the rate of new concern events change over time?
    • How does the Collision Probability change over time?
  • Data Preparation:

    • Summarize by each event to show the Collision Probability for each fragment size at the last recorded time, if that time is within a day of TCA.

    • Summarize by each event to show the Collision Probability for each fragment size at the last recorded time, for each binned days to TCA.


Collision Probability Density

We need to find a range of values to appropriately label events of concern.

How many events have \(\text{Collision Probability} = 0\) when \(\text{days to TCA} < 1\)?

Figure 1: Collision Probability within One Day of TCA

We use,
\(\text{Collision Probabilty} >= 1\text{e-}5 \text{ and days to TCA} < 1 \text{ day,}\)


as a surrogate to label a concern event.

Concern Event Frequency

How is our visibility of concern events limited?

How many Concern Events if the threshold is \(1\text{e-}5\)?

We focus our initial analysis on concern events with fragment size of \(>= 1\) fragment. Future methodology will include events with higher fragment sizes.

Figure 2: Concern Event Count over Time

Conclusions:

  • At \(5 \text{ days to TCA}\) the number of new concern events drastically increases.
  • We will only have visibility of about \(50%\) of concern events at \(5 \text{ days to TCA}\).
  • There is an advantage to a warning time interval within 5 days.

Collision Probability Variability

  • Does the \(\text{Collision Probability}\) increase as we aproach \(\text{TCA}\)?
  • Should we expect to have a lower warning threshold at \(5 \text{ days to TCA}\)

Figure 3: Collision Probability as TCA Varies

Conclusions:

  • The variability expands slightly, but there is not overwhelming evidence of an increase in the \(\text{Collision Probabilty}\).
  • We consider the following range for \(\text{Collision Probability}\) warning thresholds:

\(1.94\text{e-}22 <= Pc\_warn <= 1\text{e-}5\),


where \(Pc\_warn\) is the \(\text{Collision Probability}\) that triggers a warning.

Risk Tradespace

We need to evaluate warning thresholds by examining the trade space between risk aversion and tolerance.

We have the following working definitions:


\(\text{Concern Event} := \text{Collision Probability} <= 1\text{e-}5 \text{ and days to TCA < 1 day}\)

\(\text{FN} := \text{Number of Concern Events that did not trigger a warning at 5 days to TCA}\)

\(\text{FP} := \text{Number of false alarms, where warned events at 5 days to TCA did not become Concern Events}\)


Figure 4: Confusion Rates at Five Days to TCA and Concern Prop = 1e-5

Hovering over the plot, we see the that the warning threshold decreases as FN decreases and as FP increases. In this case, the more severe mistake is FN (not warning for a concern event), which is our Type II error, and the FP (false alarm, Type I error) has much lower rates. The FP rates are so low because of the high number of events with Collision Probabilities falling below \(1\text{e-}5\) within one day to TCA.

We recommend this chart to inform warning thresholds for five days to TCA, when a concern event is defined above.

Way Forward

  • Finalize warning threshold point estimate formulation.
  • Employ Monte Carlo method to explore the variability of the point estimate, and deliver a \(95% \text{bootstrap confidence inverval}\) for warning threshold at five days to TCA.
  • Repeat this methodology for 4, 3, 2 days to TCA.
  • Iterate on the R Shiny Application based on client feedback.